Papers with sample efficiency of
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to theorem proving in large language models rely on value functions and/or Monte Carlo Tree Search (MCTS), but the potential of simpler methods like Best-First Tree Search remains underexplored. |
| Approach: | They propose a scalable expert iteration framework that implements strategic data filtering at each expert iteration round, excluding problems solvable via beam search node expansion to focus on harder cases. |
| Outcome: | The proposed framework achieves a state-of-the-art score of 72.95 on the MiniF2F test set and challenges the perceived necessity of complex tree search methods. |